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(57) Abstract 

A method and apparatus for editing electronic ink for pen-based 
computers in which the ink is represented in stroke form. The invention 
implements a "backspace" ink function, similar to the backspace key on 
a keyboard. In the preferred embodiment, selecting a "backspace" ink 
icon (60) with the pen (3) of a pen-based computer (1), or pressing a 
hardware button on the computer, erases the last stroke, or segment of a 
stroke, drawn by the user. Erasure is accomplished by deleting (63) the 
last stroke, or segment of a stroke, in a sequential list of strokes (Figure 
3) representing the user's input. Again selecting the backspace icon 
(65) erases me previous line stroke or stroke segment. In an alternative 
embodiment, any displayed line segment of a drawing can be selected 
as the starting point for erasing, and erasing can be done on a stroke- 
by-stroke or segment-by-segment basis, either "forward" or "backward" 
from the selected stroke. 
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METHOD AND APPARATUS FOR EDITING ELECTRONIC INK 



BACKGROUND OF THE INVENTION 

1. Field of the Invention 

5 This invention relates to computers, and more 

particular, to a method and apparatus for editing 
electronic ink for pen-based computers. 

2. Description of Related Art 

10 As computers have become increasingly popular for 
various applications, portable computers have been 
developed for a wide variety of uses. While many such 
portable computers use a traditional keyboard for 
input, for smaller computers, particularly including 

15 hand-held computers, the use of "pens" as an interface 
has been introduced as a way of making a small computer 
easier to use. With a pen interface, a user can place 
a pen or stylus directly on a touch-sensitive screen of 
the computer to control the software running on the 

20 computer and to input information. For many people, 
writing with a pen is a more natural way of inputing 
information than using a keyboard. 



An example of a prior art pen-based hand-held computer 
25 is shown in FIGURE 1. The illustrated hand-held 

computer 1 is typically about 4 inches by 6.5 inches, 
with a substantial part of one surface comprising a 
touch-sensitive display screen 2. The display screen 2 
is typically a liquid crystal display (LCD) having a 
30 resolution of 240x320 pixels or greater (although 

larger or smaller pixel arrays could be used) . Various 
technologies can be used to sense and digitize the 
location of a pen or stylus 3 touched against the 
surface of the LCD screen 2 to indicate to the compu- 
35 ter's operating system the X-Y coordinates of the 

touch. Various hardware buttons 4 may be provided to 
control different functions, and/or to turn power on or 



2 . 

off to the unit. In addition, a variety of software 
buttons or icons 5 may be provided, in known fashion, 
to indicate such functions as, for example, word 
processing or a calendar. 

5 

An inherent characteristic of such pen-based computers 
is the use of electronic "ink" to depict input by a 
user on the touch-sensitive display screen 2. "Ink" 
comprises a series of pixels changed (e.g., darkened or 

10 lightened) as a pen 3 is moved across the touch- 
sensitive display screen 2 by a user, thus mimicking 
the application of real ink to paper. The ink input is 
typically represented internally as one or more 
"strokes", each comprising a set of points which can be 

15 connected on the display screen 2 to give the illusion 
of continuous lines. 

In many systems, characters drawn in electronic ink can 
be recognized and converted to ASCII text. One such 

20 character recognition system is described in U.S. 

Patent No. 5,125,039, entitled "Object Recognition 
System", by the inventor of the present invention. In 
such systems, as a user inputs each ink character, or 
after several ink characters have been input, the 

25 computer applies algorithms to recognize each character 
as an ASCII character, and replaces the ink 
representation of the character drawn by the user with 
a standardized ASCII representation of that character. 
In other cases, it is desirable to leave the electronic 

30 ink unrecognized. 

Editing ink input has proven difficult. Two prior 
methods have generally been used for editing electronic 
ink input. The first and most common method is to use 
35 the pen 3 as an electronic "eraser". In this method, 
the user touches an eraser icon to enter an erasing 
mode. Thereafter, any pixel of the display screen 2 



WO 94/16408 



PCT/US93/12239 



3. 

that contains electronic ink input is erased if the pen 
3 passes over that pixel. This method works very much 
like a real eraser, and has several similar problems. 
It is difficult to accurately erase only desired 
5 portions of a figure, and nothing else. Thus, it is 

common to accidentally erase part of a figure that was 
intended to be kept, requiring the user to go back and 
redraw the partially erased line after changing back to 
a drawing mode. The problem is exacerbated by the 

10 inaccuracies of the digitizer, and inaccuracies and 
parallax caused by the spacing of the pen 3 from the 
actual display screen 2 by the glass covering of the 
display screen 2, both of which makes it difficult to 
both erase accurately and to fix any inadvertently 

15 erased lines. 

Another problem with the eraser method is that it leads 
to a drawing made up of many small strokes, some of 
which may be difficult to see. Further, what 
20 originally appeared to be smooth curves as drawn fre- 
quently end up as uneven curves made up of multiple 
line segments after erasing and then fixing parts of 
the original curve. 

25 The second method for editing electronic ink is to have 
the user select a block of ink, designated by means of 
a selection rectangle, "lasso" feature, or other means. 
The user can then move, copy, or delete the selected 
portion of the ink figure. The biggest problem with 

3 0 this method is accurately selecting only the ink to be 
erased. The user may often select lines or parts of 
lines that the user did not intend to change, requiring 
that the user fix the erased portions. 

35 In addition to the above drawbacks of the prior art, 

both methods of editing electronic ink make it unlikely 
that edited ink can be later recognized as ASCII text. 
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Most handwriting recognition algorithms become confused 
when strokes are moved, partially erased, or consist of 
many small segments. All of these defects are likely 
to occur with existing methods of editing electronic 
5 ink. 

Accordingly, it would be desirable to have an improved 
method for editing electronic ink, which permits easy 
and accurate selection of only those portions of a 
10 drawing that a user desires to erase and the ability to 
erase a selected portion without affecting other 
portions of a displayed figure. The present invention 
provides such a solution. 

15 SUMMARY OP THE INVENTION 

The invention comprises a method and apparatus for 
editing electronic ink for pen-based computers in which 
the ink is represented in stroke form. The invention 
implements a "backspace" ink function, similar to the 

20 backspace key on a keyboard. In the preferred 

embodiment of the invention, selecting a "backspace" 
ink icon with the pen of a pen-based computer, or 
pressing a hardware button on the computer, erases the 
last stroke, or segment of a stroke, drawn by the user. 

25 Erasure is accomplished by deleting the last stroke, or 
segment of a stroke, in a sequential list of strokes 
representing the user's input. Again selecting the 
icon or pressing the button erases the previous line 
stroke or stroke segment, and so on. This method 

30 allows the user to accurately erase any amount of 
electronic ink represented in stroke form without 
disrupting nearby or overlapping lines. 

In an alternative embodiment of the invention, any dis- 
35 played line segment of a drawing can be selected as the 
starting point for erasing, and erasing can be done on 
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a stroke-by- stroke or segment-by-segment basis, either 
"forward" or "backward" from the selected stroke. 

The details of the preferred embodiment of the present 
5 invention are set forth in the accompanying drawings 
and the description below. Once the details of the 
invention are known, numerous additional innovations 
and changes will become obvious to one skilled in the 
art. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 is a front left-side perspective drawing 
showing a prior art pen-based computer. 

15 FIGURE 2 is a diagrammatic example of electronic ink 

strokes made on a digitizer/display grid of a pen-based 
computer, showing a single-stroke curved line and a 
double-stroke character. 

20 FIGURE 3 is a representation of the stroke data for the 
single-stroke curved line and the double-stroke charac- 
ter of FIGURE 2. 

FIGURE 4 is a diagrammatic example of FIGURE 2, showing 
25 the single-stroke curved line after erasure of one 
stroke of the double-stroke character. 

FIGURE 5 is a diagrammatic example of FIGURE 2, showing 
the single-stroke curved line after erasure of both 
30 strokes of the double-stroke character. 

FIGURE 6 is a flow-chart of the preferred embodiment of 
the inventive method. 

35 Like reference numbers and designations in the various 
drawings refer to like elements. 
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DETAILED DESCRIPTION OF THE INVENTION 

Throughout this description, the preferred embodiment 
and examples shown should be considered as exemplars, 
rather than limitations on the present invention. 

5 

FIGURE 2 is a diagrammatic example of electronic ink 
strokes made on a digitizer/display grid of a pen-based 
computer, showing a single-stroke curved line 10 and a 
character "D" 12 comprising two strokes 14, 16. FIGURE 

10 3 is a representation of the stroke data for the 

single-stroke curved line 10 and the double-stroke 
character 12 of FIGURE 2. As can be seen from FIGURE 
3, each stroke is represented by a set of paired 
numbers, each pair comprising the Cartesian coordinates 

15 of a pixel. Adjacent pairs of numbers define a segment 
of the stroke. (A typical input from a digitizer would 
contain more points than shown; FIGURE 2 is simplified 
for clarity) . 

20 A numeric representation of a stroke requires at least 
two pairs of coordinates. As FIGURE 3 shows, the 
single-stroke curved line 10 can be described as 
comprising six pairs of pixel coordinates (i.e., five 
segments) . In the illustrated embodiment, the coordi- 

25 nate value "0,0" is used as a delimiter to designate 
the end of a series of coordinate pairs defining a 
single stroke; however, other values or means for 
delimiting strokes may be used. 

30 Alternative ways can be used to represent strokes. For 
example, the origin for the grid need not be in the 
"lower left" corner, as is typical of a standard 
mathematical representation of a Cartesian grid, but 
may be in the "upper left" , which is more typical of 

35 the addressing used for a pixel array. As another 

example, the coordinate points representing a stroke 
can be compressed, or the strokes can be represented as 
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mathematical equations (such as the equation for a line 
or a curve) , or in any other desired form. The 
invention will work with any representation of strokes, 
such that each stroke can be distinguished from each 
5 other stroke. Thus, the term "stroke", as used herein, 
should be understood to mean any non-bitmapped 
representation of lines or figures input by a user as 
electronic ink. 

10 The stroke data may be stored in a computer memory as 
simply a list of stroke entries in order of creation, 
each separated by a delimiter. Alternatively, the 
stroke data may be stored in random order in a computer 
memory as a linked list, in known fashion. As another 

15 alternative, the stroke data may be stored in random 

order in a computer memory with a list of sequential or 
linked pointers being used to indirectly define an 
entry order for the stroke data. Any data structure 
that permits input strokes to be accessed at least in 

20 order (or reverse order) of entry can be used to store 
the stroke data. Because the strokes comprising each 
drawing are accessible in the sequential order of 
input, it is easy to determine the "last" stroke drawn 
by the user. Hence, the term "stroke list" should be 

25 interpreted to cover any format from which the 

sequential order of input of strokes can be determined. 

To display the input strokes, the list of strokes is 
read from the computer memory. For each stroke, the 
30 intensity of the pixels comprising each consecutive 
pair of coordinate pairs is changed, along with all 
interpolated pixels in between, in known fashion. 

Thus, in accordance with these definitions, the data 
3 5 structure sequence shown in FIGURE 3 shows that the 
stroke for the curved line 10 in FIGURE 2 was drawn 
before the vertical line stroke 14 for the character 
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"D" 12, which in turn was drawn before the curved line 
stroke 16 of the character "D" 12. When the strokes 
are read from the stroke list, the display of the 
computer will appear as in FIGURE 2 (without the grid 
5 lines) . 

To accomplish the "backspace" ink function of the 
present invention, the computer detects the touch of a 
pen on a "backspace" ink icon, or the press of a 

10 "backspace" hardware button on the computer, by the 
user. The list of strokes stored in the computer is 
then examined to determine the last entered stroke. 
Since the strokes are accessible in sequential order, 
the last stroke can easily be determined by sequen- 

15 tially examining the entire list of strokes until the 
last "0,0" delimiter is accessed, or by keeping track 
(e.g., by means of a pointer) of the beginning address 
of the last stroke entered. Of course, other methods 
for determining the last entered stroke could also be 

20 used. 

Once the last entered stroke is located, that stroke 
entry is simply deleted from the stroke list in the 
computer memory so that the next prior entry becomes 

25 the "last" stroke. Thus, referring to FIGURE 4, when 
the "backspace" ink function is selected, the stroke 
for the curved line portion 16 of the character "D" is 
deleted. Thereafter, the display screen is redrawn to 
show only the remaining strokes, comprising the single- 

30 stroke curved line 10 and the vertical line portion 14 
of the character "D" 12. Since the displayed lines are 
re-drawn based upon the current contents of the stroke 
list, any pixels in common between a deleted stroke and 
other strokes are also redrawn. Thus, deletion of a 

35 stroke has no effect on the appearance of the remaining 
strokes . 
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Again selecting the icon or pressing the "backspace" 
ink button erases the previous stroke in the stroke 
list, and so on. Thus, in FIGURE 5, pressing the 
"backspace" ink function again causes the vertical line 
5 stroke 14 to be deleted in the manner described above. 

If desired, after the "backspace" ink function is 
selected, the last stroke may be highlighted, in known 
fashion (e.g., by holding or blinking the stroke), and 

10 the user prompted for approval of the deletion of that 
stroke. Also, if the user is prompted for a deletion, 
but reselects the "backspace" ink function, the prior 
stroke in the stroke list can be highlighted, and the 
user prompted for approval of the deletion of that 

15 stroke. In like manner, further reselection of the 
"backspace" ink function highlights the next prior 
stroke in the stroke list for prompted deletion. 

FIGURE 6 is a flow-chart of the preferred embodiment of 
2 0 the inventive method. The flow chart shows the 
following steps in abbreviated form: 

1. Test if the pen 3 is in a "backspace ink" icon, or 
a backspace button has been pressed (Step 60) . 

2. If no, loop back to start (Step 61). 
25 3. If yes, then: 

3.1 Find the last entry in the list of strokes 
(Step 62) . 

3.2 Delete the found entry (Step 63). 

3.3 Redisplay the remaining strokes (Step 64). 
30 4. Go back to start (Step 65). 

Of course, other step sequences may be used to accom- 
plish the same function. 

In an alternative embodiment of the invention, any dis- 
35 played stroke of a drawing can be selected as the 

starting point for erasing, and erasing can be done on 
a stroke-by-stroke basis, either "forward" or "back- 
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ward" from the selected stroke. This is most easily 
accomplished by permitting a user to select with the 
pen 3 any desired displayed stroke, and setting a 
pointer to the sequential position of that stroke in 
5 the stroke list. If the "backspace ink" function is 
then selected, the entry of the selected stroke is 
deleted from the stroke list. The pointer is then set 
to point to the entry just prior in sequence to the 
deleted entry. If the "backspace ink" function is 

10 selected again, the entry pointed to by the pointer is 
deleted, and the pointer reset to the prior entry in 
the stroke list. To delete in a "forward direction", 
the user selects a different icon or button, and the 
entry in the stroke list just after the pointer is 

15 deleted. In an alternative embodiment, after a 

deletion, the pointer is set to point to the entry just 
after the deleted entry, and deletions in either 
direction are based upon that position as a reference 
point. In either case, the stroke indicated by the 

20 . pointer may be highlighted and the user prompted for 
further deletions. 

In another alternative embodiment of the invention, ink 
is erased segment-by-segment instead of stroke-by- 

25 stroke. In this method, pressing a "backspace ink" 

icon causes ink to be erased one segment at a time in 
either a forward or backward direction, starting either 
from the end of the last stroke or from a user-selected 
point. For example, referring to FIGURE 3, if the last 

30 stroke (the curved line portion 16 of the character "D" 
shown in FIGURE 2) is selected for erasure, the last 
segment (defined by coordinates 11,2 and 9,1) would be 
deleted by deleting the last coordinate pair (i.e., 
9,1) and moving up the stroke delimiter (i.e., 0,0). 

35 This method allows even finer control of exactly which 
ink input is erased, while maintaining the benefit of 
not erasing nearby or adjacent strokes. 
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Because the lines of a drawing are erased on a stroke- 
by-stroke or segment-by-segment basis, the inventive 
apparatus and method allows the user to accurately 
erase any amount of electronic ink represented in 
5 stroke form without disrupting nearby or overlapping 
strokes. The invention provides a means for editing 
electronic ink which permits accurate selection of only 
those portions of a drawing that a user desires to 
erase and the ability to erase a selected portion 
10 without affecting other portions of a displayed figure. 

A number of embodiments of the present invention have 
been described. Nevertheless, it will be understood 
that various modifications may be made without 
15 departing from the spirit and scope of the invention. 

Accordingly, it is to be understood that the invention 
is not to be limited by the specific illustrated 
embodiment, but only by the scope of the appended 
claims. 
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A method for editing electronic ink for a pen-based 
computer in which the ink is represented as a 
plurality of entries in a stroke list in a computer 
memory, and displayed on a display screen of the 
computer, comprising the steps of: 

(a) accepting input from a user selecting a 
deletion function; 

(b) deleting the last entry from the stroke list in 
response to the accepted input. 

A method for editing electronic ink for a pen-based 
computer in which the ink is represented as a 
plurality of entries in a stroke list in a computer 
memory, and displayed on a display screen of the 
computer, comprising the steps of: 

(a) accepting input from a user selecting a 
deletion function; 

(b) finding the last entry in the stroke list in 
response to the accepted input; 

(c) deleting the found last entry from the stroke 
list; 

(d) displaying the remaining entries of the stroke 
list on the display screen of the computer. 

A method for editing electronic ink for a pen-based 
computer in which the ink is represented as a 
plurality of entries in a stroke list in a computer 
memory, and displayed on a display screen of the 
computer, each entry comprising at least one 
segment, comprising the steps of: 

(a) accepting input from a user selecting a 
deletion function; 

(b) deleting the last segment of the last entry 
from the stroke list in response to the 
accepted input. 
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4. A method for editing electronic ink for a pen-based 
computer in which the ink is represented as a 
plurality of entries in a stroke list in a computer 
memory, and displayed on a display screen of the 
5 computer, each entry comprising at least one 

segment, comprising the steps of: 

(a) accepting input from a user selecting a 
deletion function; 

(b) finding the last entry in the stroke list in 
10 response to the accepted input; 

(c) deleting the last segment of the found last 
entry from the stroke list; 

(d) displaying the entries of the stroke list on 
the display screen of the computer. 



5. A method for editing electronic ink for a pen-based 
computer in which the ink is represented as a 
plurality of entries in a stroke list in a computer 
memory, and displayed on a display screen of the 

5 computer, comprising the steps of: 

(a) accepting pen input from a user selecting a 
displayed stroke; 

(b) accepting input from a user selecting a 
deletion function; 

L0 (c) deleting the selected displayed stroke from the 

stroke list in response to the accepted inputs. 

6. The method for editing electronic ink of claim 5, 
further including the steps of: 

(a) accepting further input from a user selecting 
the deletion function; 
5 (b) deleting the displayed stroke adjacent to the 

last deleted stroke. 
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A method for editing electronic ink for a pen-based 
computer in which the ink is represented as a 
plurality of entries in a stroke list in a computer 
memory, and displayed on a display screen of the 
computer, each entry comprising at least one 
segment, comprising the steps of: 

(a) accepting pen input from a user selecting a 
displayed stroke; 

(b) accepting input from a user selecting a 
deletion function; 

(c) deleting the last segment of the selected 
displayed stroke from the stroke list in 
response to the accepted inputs. 

The method for editing electronic ink of claim 7, 
further including the steps of: 

(a) accepting further input from a user selecting 
the deletion function; 

(b) deleting the segment adjacent to the last 
deleted segment. 

ft method for editing electronic ink for a pen-based 
computer in which the ink is represented as a 
plurality of entries in a stroke list in a computer 
memory, and displayed on a display screen of the 
computer, comprising the steps of: 

(a) accepting pen input from a user selecting a 
displayed stroke; 

(b) accepting input from a user selecting a 
deletion function; 

(c) finding the entry of the selected displayed 
stroke in the stroke list in response to the 
accepted inputs; 

(d) deleting the found selected entry from the 
stroke list; 

(e) displaying the remaining entries of the stroke 
list on the display screen of the computer. 
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10. The method for editing electronic ink of claim 9, 
further including the steps of: 

(a) accepting further input from a user selecting 
the deletion function; 

(b) deleting the entry in the stroke list adjacent 
to the entry of the last deleted stroke. 

11. A method for editing electronic ink for a pen-based 
computer in which the ink is represented as a 
plurality of entries in a stroke list in a computer 
memory, and displayed on a display screen of the 
computer, each entry comprising at least one 
segment, comprising the steps of: 

(a) accepting pen input from a user selecting a 
displayed stroke; 

(b) accepting input from a user selecting a 
deletion function; 

(c) finding the entry of the selected displayed 
stroke in the stroke list in response to the 
accepted inputs; 

(d) deleting the last segment of the found entry 
from the stroke list; 

(e) displaying the entries of the stroke list on 
the display screen of the computer. 

12. The method for editing electronic ink of claim 11, 
further including the steps of: 

(a) accepting further input from a user selecting 
the deletion function; 

(b) deleting the segment adjacent to the last 
deleted segment. 
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The method for editing electronic ink of claims 1, 
2, 3, 4, 5, 7, 9, or 11, further including the step 
of prompting the user to approve each deletion. 
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The method for editing electronic ink of claim 13, 
further including the step of highlighting the 
displayed stroke corresponding to the item to be 
deleted while prompting the user. 
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